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BASIC  ATTRIBUTES  TEST  (BAT);  A  PRELIMINARY  COMPARISON 
BETWEEN  RESERVE  OFFICER  TRAINING  CORPS  (ROTC) 
AND  OFFICER  TRAINING  SCHOOL  (OTS)  PILOT  CANDIDATES 


I.  INTRODUCTION 

The  Basic  Attributes  Test  (BAT)  battery  Is  a  set  of  computer-administered  personnel  tests 
developed  for  Individual  administration  under  standardized  conditions  (Carretta,  1987a).  The  BAT 
battery  is  designed  to  provide  measures  of  psychomotor  skills,  cognitive  and  perceptual  abilities, 
personality  characteristics,  and  attitudes  believed  to  be  related  to  United  States  Air  Force  (USAF) 
flying  training  performance  (Imhoff  &  Levine,  1981). 

Since  1983,  when  administration  of  the  BAT  to  USAF  pilot  candidates  began,  the  original 
IS  tests  have  been  evaluated  in  terms  of  their  utility  for  predicting  flying  training  performance 
(Carretta,  1988b:  Siem,  1988).  Results  from  these  studies  suggest  that  performance  Information 
from  a  subset  of  the  original  15  tests  provides  the  greatest  gains  in  predictive  validity  when 
combined  with  the  Air  Force  Officer  Qualifying  Test  (AFOOT),  the  personnel  test  currently  used 
to  provide  Individual  abilities  information  for  aircrew  selection  and  classification. 


Purpose 

The  purpose  of  the  present  investigation  was  to  develop  an  interim  scoring  profile  for  BAT 
score  interpretation  based  on  pilot  candidates  commissioned  through  the  Reserve  Officer  Training 
Corps  (ROTC)  and  the  Officer  Training  School  (OTS)  programs.  Only  those  eight  BAT  tests 
which  have  demonstrated  the  greatest  potential  for  adding  to  the  predictive  validity  of  the  AFOOT 
are  treated  In  detail  here.  Table  1  provides  a  brief  description  of  these  tests.  A  more  detailed 
description  Is  provided  later.  For  completeness,  descriptions  of  the  remaining  seven  tests  are 
provided  in  Appendix  B. 


II.  METHOD 
Subjects 

The  subjects  In  this  study  were  USAF  officer  candidates  from  the  ROTC  (N  =  350)  and 
OTS  (N  =  705)  commissiorilng  programs  who  were  tested  on  the  AFOOT  and  the  BAT.  These 
subjects  had  already  been  chosen  for  Undergraduate  Pilot  Training  (UPT),  in  part  on  the  basis 
of  their  AFOOT  scores.  Subjects  were  Informed  that  the  study  Involved  performing  experimental 
computerized  tests  being  considered  for  operational  use.  They  also  were  told  their  performance 
would  not  affect  their  status  in  the  program,  would  be  kept  confidential,  and  would  be  used 
only  for  research  purposes.  Not  all  subjects  had  scores  on  all  eight  BAT  tests  considered  here 
because  of  changes  in  the  test  composition  of  the  BAT  battery  over  the  administration  period. 
Demographic  information  regarding  the  ROTC  and  OTS  samples  is  provided  in  Table  2. 


Tests 


Air  Force  Officer  Qualifying  Test 

AFOOT  Form  0  is  a  paper-and-pencil  test  battery  that  consists  of  16  subtests  which  produce 
five  composite  percentile  scores;  verbal,  quantitative,  academic  aptitude  (verbal  and  quantitative 
combined),  pilot,  and  navigator-technical.  Of  these  five  composites,  only  two-the  pilot  and 
navigator-technical  composites-are  used  in  the  operational  selection  of  pilot  candidates  (U.S.  Air 
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Table  1 .  Basic  Attributes  Test  (BAT)  Battery  Summary 
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Force,  1983).  AFOOT  subtest  scores  are  usually  represented  as  raw  test  scores,  whereas  the 
composite  Indices  are  percentile  scores  that  may  range  from  01  to  99  percent. 


Table  2.  Demographic  Data  for  ROTC  and  OTS  Samples 


Variable 

ROTC  (N  =  350)  OTS  (N  =  705) 

Sex  (%):  Male 

99.4  [  99.4] 

98.9  [98.9] 

Female 

0.6  [  0.6] 

1.1  [  1.1] 

Race  (%):  Black 

0.0  [  0.0] 

1.3  [  1.5] 

White 

11.4  [100.0] 

85.5  [98.5] 

Missing  data 

88.6 

13.2 

Age  (mean  no.  of  yrs): 

20.5 

24.0 

Not».  Many  subjects  did  not  have  valid  data  for  race.  The  percentages 
in  brackets  indicate  the  percentage  of  the  sample  when  only  cases  with 
valid  data  are  considered. 


In  Appendix  A,  Table  A-1  provides  a  brief  description  of  the  16  subtests  included  in  AFOOT 
Form  0:  Table  A-2  shows  the  subtest  composition  of  each  of  the  five  AFOOT  composites.  Table 
3  summarizes  AFOOT  composite  and  subtest  score  comparisons  between  the  ROTC  and  OTS 
samples. 


Table  3.  Summary  of  AFOOT  Comparisons 
Between  ROTC  and  OTS  Samples 


Variable 

ROTC  (N 
Mean 

=  350) 
SD 

OTS  (N 
Mean 

=  705) 
SD 

Two-tailed 

t-test 

AFOOT  Composite  Percentile 
Verbal 

Scores: 

60.5 

22.1 

68.9 

20.3 

-6.14** 

Quantitative 

66.3 

19.2 

67.5 

20.2 

-0.88 

Academic  Aptitude 

64.3 

20.6 

70.0 

18.7 

-4.53** 

Pilot 

72.1 

15.0 

78.0 

14.7 

-6.09** 

Navigator-Technical 

71.8 

16.5 

75.00 

16.1 

-2.99** 

AFOOT  Subtest  Scores: 

Verbal  Analogies 

15.8 

2.8 

16.3 

2.8 

-2.90** 

Arithmetic  Reasoning 

13.7 

3.4 

14.3 

3.5 

-2.66** 

Reading  Comprehension 

18.6 

4.3 

19.9 

3.6 

-5.19** 

Data  Interpretation 

13.8 

3.0 

14.7 

3.1 

-4.56** 

Word  Knowledge 

14.6 

4.8 

16.8 

4.6 

-7.07** 

Math  Knowledge 

19.0 

4.1 

18.3 

4.6 

2.44* 

Mechanical  Comprehension 

11.9 

2.9 

13.0 

2.9 

•5.97** 

Electrical  Maze 

10.4 

4.2 

10.6 

4.3 

-0.47 

Scale  Reading 

24.9 

4.8 

26.1 

4.8 

-3.96** 

Instrument  Comprehension 

14.1 

3.8 

14.6 

3.9 

-2.00* 

Block  Counting 

13.7 

3.3 

13.9 

3.4 

-0.97 

Table  Reading 

31.1 

5.1 

32.0 

5.5 

-2.43* 

Aviation  Information 

11.0 

3.6 

13.4 

3.9 

•9.63** 

Rotated  Blocks 

10.2 

2.4 

9.9 

2.6 

1.81 

General  Science 

10.2 

3.4 

11.0 

3.4 

-3.31** 

Hidden  Figures 

11.3 

2.3 

11.2 

2.3 

0.33 

*p  S  .05. 

**p  S  .01. 
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Basic  Attributes  Test 


Each  of  the  tests  in  the  BAT  battery  was  adapted  from  tests  in  the  research  iiterature  that 
were  identified  as  potentiaily  usefui  predictors  of  fiying  performance  (Imhoff  &  Levine,  1981). 
The  criteria  used  to  select  the  tests  included  feasibility,  interest  of  the  test-taker,  independence 
from  other  tests  in  the  battery,  construct  validity,  and  minimal  dependence  on  verbal  materials 
for  administration.  The  BAT  battery  was  designed  to  measure  a  variety  of  psychomotor  skills, 
information  processing  abilities,  and  personality  characteristics  that  are  considered  important  in 
determining  the  suitability  of  a  candidate  for  flight  training.  A  summary  of  the  most  promising 
pilot  candidate  selection  and  classification  tests  from  the  BAT  battery  is  provided  in  Table  I.  It 
indicates  the  name,  length,  attribute  measured,  measures  of  interest  and  origin  for  each  test. 
Individual  test  summaries  are  presented  below.  Illustrations  of  BAT  test  items  are  provided  in 
Appendix  C,  and  detailed  scoring  procedures  are  provided  in  Appendix  D.  BAT  apparatus 
specifications  and  details  regarding  hardware  and  software  development  are  provided  in  Carretta 
(1967a). 

Test  Battery  Introduction.  This  interactive  subprogram  prompts  the  subject  to  provide 
background  Information  (e.g.,  identity,  age,  gender),  as  well  as  personal  history  and  attitudes 
related  to  flying. 

Two-Hand  Coordination.  Two  tests  are  used  to  evaluate  psychomotor  abilities.  The  first, 
the  Two-Hand  Coordination  Test,  is  a  rotary  pursuit  task.  An  airplane  (target)  moves  in  an 
elliptical  path  on  the  screen  at  a  rate  of  20  cycles  per  minuto.  The  rate  of  movement  of  the 
airplane  within  each  cycle  varies  in  a  fixed  sinusoidal  pattern.  The  subject  controls  the  movement 
of  a  small  "gunsight"  using  a  left-hand  joystick  for  vertical  movement  of  the  gunsight  and  a 
right-hand  joystick  for  horizontal  movement  of  the  gunsight.  The  subject’s  task  Is  to  keep  the 
gunsight  on  the  moving  airplane.  After  receiving  Instructions,  the  subject  completes  a  3-minute 
practice  session  and  a  5-minute  test.  The  measures  of  interest  are  horizontal  and  vertical  tracking 
error  scores.  The  psychological  factors  assessed  in  the  Two-Hand  Coordination  Test  are 
low-to-moderate-order  tracking  and  time-sharing  ability  in  pursuit. 

Complex  Coordination.  The  Complex  Coordination  Test  uses  a  dual-axis  joystick  (right-hand 
joystick)  to  control  the  horizontal  and  vertical  movement  of  a  cursor.  The  left-hand  single-axis 
joystick  controls  the  horizontal  movement  of  a  "rudder  bar"  at  the  base  of  the  screen.  The 
subject's  task  is  to  keep  the  cursor  (against  a  constant  horizontal  and  vertical  rate  bias)  centered 
on  a  large  cross  fixed  at  the  center  of  the  screen,  while  simultaneously  centering  the  rudder 
bar  at  the  base  of  the  screen  (also  against  a  constant  rate  bias).  The  instructions,  practice, 
testing,  and  scoring  are  the  same  as  those  In  the  Two-Hand  Coordination  Test.  The  Complex 
Coordination  Test  assesses  compensatory  tracking  ability  involving  multiple-axis  continuous  events. 

Encoding  Speed.  Two  letters  are  presented  simultaneously  to  the  subject,  who  is  required 
to  make  a  same-different  judgment  about  the  letter  pair.  The  judgment  may  be  based  on  a 
Physical  identity  rule  (look  the  same  or  look  different,  AA  versus  Aa),  a  Name  identity  rule  (same 
name  or  different  name,  AA  versus  AH)  or  Category  Identity  rule  (vowels  versus  consonants  - 
AE  versus  AH).  The  reaction  time  for  the  judgment  provides  a  measure  of  the  speed  of  the 
cognitive  encoding  process. 

Reaction  time  and  accuracy  of  response  (correct/incorrect)  are  recorded  on  each  of  the  96 
trials  (32  trials  in  each  rule  condition).  The  psychological  factor  Involved  in  this  test  Is  verbal 
classification  at  several  levels  of  cognitive  operation. 

Mental  Rotation.  A  pair  of  letters  are  presented  sequentially,  and  the  subject  Is  required  to 
make  a  same-different  judgment.  Elements  of  the  letter  pair  may  be  either  identical  or  mirror 
Images,  and  the  letters  may  be  either  in  the  same  orientation  or  rotated  In  space  with  respect 
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to  each  other.  A  correct  "different"  judgment  occurs  when  one  element  of  a  pair  is  a  mirror 
Image  of  the  other,  regardless  of  orientation. 

To  take  the  test,  the  subject  must  form  a  mental  Image  of  the  first  letter  (no  longer  displayed) 
and  perform  a  point-by-point  comparison  with  the  second  letter  (which  remains  on  the  screen). 
In  addition,  when  the  letters  are  rotated  with  respect  to  each  other,  the  subject  must  mentally 
rotate  the  mental  image  of  one  letter  into  congruence  with  the  other  prior  to  making  the 
comparison. 

Reaction  time  and  accuracy  of  response  are  recorded  on  each  of  the  72  trials.  The 
psychological  factors  assessed  by  this  test  are  spatial  transformation  and  classification. 

Item  Recognition.  In  this  test,  a  string  of  one  to  six  digits  is  presented  on  the  scieen. 
The  string  Is  then  removed  and  followed,  after  a  brief  delay,  by  a  single  digit.  The  subject  is 
instructed  to  remember  the  initial  string  of  digits,  then  to  decide  If  the  single  digit  was  one  of 
those  presented  In  the  initial  string.  The  subject  is  Instructed  to  press  a  keypad  button  marked 
YES  If  the  single  digit  was  in  the  string,  or  press  another  marked  NO  if  the  digit  was  not  in 
the  string.  The  Instructions  Inform  the  subject  to  work  as  quickly  and  accurately  as  possible. 
Reaction  time  and  accuracy  of  response  are  recorded  on  each  of  the  48  trials.  Shoit-term 
memory  storage,  search,  and  comparison  operations  are  the  underlying  psychological  factors 
for  this  test. 

Time-Sharing.  During  a  series  of  10  1 -minute  trials,  the  subject  is  required  to  learn  a 
compensatory  tracking  task.  To  perform  this  task,  the  subject  must  anticipate  the  movement 
of  a  gunsight  on  a  screen  and  operate  a  control  stick  to  counteract  that  movement  in  order 
to  keep  the  gunsight  aligned  with  a  fixed  central  point  (an  airplane).  Task  difficulty  Is  adjusted 
throughout  the  test,  depending  on  the  subject's  performance.  The  gunsight  movement  control 
dynamics  are  a  combination  of  rate  and  acceleration  components.  The  "disturbance"  factor  is 
a  quasi-random  summed  sinusoidal  forcing  function. 

After  these  'Iracking  only"  trials,  the  subject  is  required  to  track  while  cancelling  digits  that 
appear  at  random  intervals  and  locations  on  the  screen.  A  digit  Is  cancelled  when  the  subject 
presses  the  corresponding  button  on  the  keypad.  If  the  subject  fails  to  respond  tc  a  digit 
within  4  seconds  after  its  appearance  time,  the  gunsight  will  disappear  until  a  digit  response  is 
made.  These  dual-task  trials  occur  in  two  3-minute  blocks.  The  information  processing  load 
gradually  Increases  during  these  trials.  The  Time-Sharing  Test  ends  with  a  final  3-minute  block 
of  'tracking  only"  trials.  There  are  a  total  of  19  1 -minute  trials  (10  tracking  only,  6  dual-task, 
and  3  more  tracking  only). 

The  effects  of  the  secondary  task  loads  are  reflected  in  the  pattern  of  level  of  tracking 
difficulty  changes  caused  by  the  adaptive  logic  that  holds  tracking  error  constant.  Feedback 
concerning  tracking  difficulty  Is  provided  by  a  gauge  that  appears  in  the  top  right  of  the  screen. 
The  measures  of  Interest  for  this  test  Include  the  level  of  tracking  difficulty  at  which  the  subject 
can  perform  consistently,  response  time  on  the  secondary  task,  and  dual-task  performance. 

This  test  assesses  a  variety  of  psychological  factors  including  higher-order  tracking  ability, 
and  learning  rpte  and  time-sharing  ability  as  a  function  of  differential  task  load. 

Self-Crediting  Word  Knowledge.  This  test  is  essentially  a  vocabulary  test  wherein  a  "target" 
word  is  presented  to  the  subject  along  with  five  other  words  from  which  its  closest  synonym 
must  be  chosen.  There  are  three  blocks  of  10  questions  each.  The  target  words  become 
Increasingly  difficult  with  each  successive  block.  Subjects  are  informed  of  this  increasing  difficulty 
and  are  required  to  make  a  "bet"  prior  to  each  block  as  to  how  well  they  expect  to  do. 
Response  time  and  accuracy  are  recorded  on  each  of  the  30  trials.  This  test  assesses 
self-assessment  ability  and  self-confidence. 
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Activities  Interest  Inventory.  This  test  Is  designed  to  determine  the  subject’s  Interest  In 
various  activities.  In  this  test,  81  pairs  of  activities  are  presented,  and  the  subject  Is  asked  to 
choose  between  them.  The  subject  Is  told  to  assume  that  he/she  has  the  necessary  ability  to 
perform  each  activity.  The  activity  pairs  force  the  subject  to  choose  between  tasks  tfiat  differ 
as  to  degree  of  threat  to  physical  survival  (sometimes  subtly,  sometimes  not).  The  measures 
of  interest  are  the  number  of  high-risk  options  chosen  and  the  amount  of  time  required  to 
choose  between  pairs  of  activities.  The  psychological  factor  assessed  by  this  test  Is  attitude 
toward  risk-taking. 


Apparatus 

The  Basic  Attributes  Test  (BAT)  apparatus,  shown  in  Appendix  C  (Figure  C-9),  consisted  of 
a  microcomputer  built  into  a  ruggedized  chassis  with  a  glare  shield  and  side  panels  designed 
to  eliminate  distractions.  Each  subject  responded  to  the  test  stimuli  by  using  individually,  and 
in  combination,  a  two-axis  joystick  on  the  right  side  of  the  apparatus,  a  single-axis  joystick  on 
the  left  side,  and  a  keypad  in  the  center  of  the  test  unit.  The  keypad  Included  the  number 
keys  0  to  9,  an  ENABLE  key  in  the  center,  and  a  bottom  row  with  YES  and  NO  keys  and  two 
ethers  labeled  S/L  (for  same/left  responses)  and  D/R  (for  different/right  responses). 

Procedure 


Each  subject  completed  both  the  AFOOT  and  the  BAT.  Pilot  candidates  were  commissioned 
through  either  ROTC  or  OTS.  Candidates  commissioned  through  ROTC  took  the  AFOOT  between 
their  senior  year  in  high  school  and  their  Junior  year  in  college.  They  completed  the  BAT  during 
the  summer  following  their  junior  year  in  college.  For  OTS  pilot  candidates,  the  AFOOT  was 
administered  either  during  their  final  year  in  college  or  after  the  completion  of  college  and  the 
BAT  was  administered  at  the  beginning  of  their  participation  in  a  Flight  Screening  Program  (FSP). 

After  the  test  administrator  initiated  the  BAT  system,  the  test  session  was  self-paced  for  each 
subject.  The  test  session  lasted  about  3  1/2  hours  and  Included  programmed  breaks  between 
tests  to  avoid  mental  and  physical  fatigue. 


Analysis 

Test  performance  on  the  AFOOT  by  the  ROTC  and  OTS  samples  was  evaluated  using 
descriptive  and  inferential  statistical  procedures.  The  objective  was  to  determine  whether  there 
were  significant  AFOOT  performance  differences  between  the  ROTC  and  OTS  pilot  candidates 
that  might  affect  their  performance  on  the  BAT  battery. 

Performance  on  the  eight  BAT  tests  was  examined  in  a  similar  manner  for  the  ROTC  and 
OTS  samples.  A  factor  analysis  was  performed  to  evaluate  relationships  among  the  BAT  summary 
scores. 


III.  RESULTS  AND  DISCUSSION 
Air  Force  Officer  Qualifying  Test 

As  shown  in  Table  3,  OTS  pilot  candidates  consistently  scored  higher  than  their  ROTC 
counterparts  on  the  AFOOT  composites.  The  OTS  group  scored  significantly  higher  on  four  of 
the  five  composites  and  11  of  the  16  subtests,  whereas  the  ROTC  group  scored  significantly 
higher  on  only  the  Math  Knowledge  subtest.  As  previously  noted,  ROTC  cadets  take  the  AFOOT 
between  their  senior  year  In  high  school  and  completion  of  their  Junior  year  in  college.  OTS 
candidates  usually  take  the  AFOOT  upon  completing  the  baccalaureate  degree.  The  OTS 
advantage  may  be  due  to  taking  the  AFOOT  after  having  achieved  a  higher  educational  level. 
Steuck,  (In  press)  reported  similar  score  differences  for  an  unscreened  group  of  ROTC  and  OTS 
officer  candidates. 
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These  score  differences  do  not  Influence  the  comparative  likelihood  of  selection  for  pilot 
training  for  members  of  the  two  groups,  as  ROTC  and  OTS  pilot  training  candidates  are  evaluated 
by  separate  selection  boards. 


Basic  Attributes  Test 


Descriptive  Measures 

For  all  of  the  tests  In  the  BAT  battery,  tracking  error  scores,  response  latencies,  and  response 
choice/accuracy  are  used  to  assess  Individual  differences  In  performance.  These  types  of  scores 
tend  to  exhibit  extremely  skewed,  non-normal  distributions.  It  Is  difficult  to  Interpret  summary 
statistics  from  such  distributions  as  both  means  and  standard  deviations  tend  to  be  distorted 
by  extreme  scores.  To  reduce  this  effect,  all  BAT  scores  that  were  more  than  three  standard 
deviations  from  the  mean  were  recoded  to  be  exactly  three  standard  deviations  from  the  mean. 
In  most  Instances  this  affected  only  a  few  scores;  however,  up  to  8.85%  of  the  scores  for  a 
test  were  affected.  Even  after  recoding,  some  distributions  were  skewed.  See  Tables  4  and  5 
for  detailed  descriptions  of  the  ROTC  and  OTS  BAT  score  distributions.  Estimates  of  the  internal 
consistency  of  the  test  items  are  provided  in  Appendix  D  (Table  D-1). 

ROTC  Versus  OTS  Comparisons 

Although  the  ROTC  and  OTS  groups  exhibited  differences  In  performance  on  the  BAT  battery, 
the  direction  of  the  differences  did  not  clearly  favor  one  group  over  the  other.  Table  6  summarizes 
comparisons  between  the  ROTC  and  OTS  group  mean  scores. 

The  two  groups  did  not  differ  in  a  consistent  manner  In  tracking  performance.  The  OTS 
group  had  marginally  lower  X2  tracking  error  scores  on  the  Complex  Coordination  Test  (M  ROTC 
=  9,497.5,  M  OTS  =  8,421.0;  t  [1053]  =  2.29,  £  £  .05).  However,  the  ROTC  group  performed 
at  a  higher  average  tracking  difficulty  on  the  compensatory  tracking  task  used  in  the  Time-Sharing 
Test  (M  ROTC  =  263.7,  M  OTS  =  256.2;  t  [1053]  =  3.19,  £  <  .01). 

Results  from  the  other  six  tests  also  were  mixed.  ROTC  subjects  made  quicker  responses 
on  two  of  the  cognitive  and  perceptual  abilities  tests  Including  Mental  Rotation  average  response 
time  (M  ROTC  =  407.0  milliseconds  (ms),  M  OTS  =  449.0  ms;  t  (1053)  =  -2.56,  £  <  .05) 
and  Tlrhe-Sharlng  average  response  time  (M  ROTC  =  1,202.8  ms,  M  OTS  =  1,238.5  ms;  t 
[1053]  =  -2.66,  £  £  .01).  OTS  subjects  achieved  a  higher  level  of  accuracy  than  did  the 
ROTC  subjects  on  three  of  the  cognitive  abilities  tests  (Encoding  Speed,  Mental  Rotation  and 
Item  Recognition),  although  this  difference  was  statistically  significant  for  only  the  Item  Recognition 
Test  (M  ROTC  =  94.3%  correct,  M  OTS  =  95.1%  correct;  t  [957]  =  -2.81,  £  s  .01). 

On  the  two  personality-type  BAT  tests,  ROTC  subjects  required  more  time  to  make  decisions 
and  were  less  willing  to  take  risks  than  were  the  OTS  subjects.  For  instance,  on  the  Self-Crediting 
Word  Knowledge  Test,  a  test  of  self-confidence,  ROTC  subjects  took  longer  to  make  decisions 
(average  response  time:  M  ROTC  =  7,812.2  ms,  M  OTS  =  7,592.5  ms;  t  [1053]  =  2.30,  p 
s  .05)  and  had  lower  expectations  about  their  performance  (bet  less;  M  ROTC  =  38.0,  M  OTs 
=  40.2;  t  [1053]  =  -4.26,  £  s  .01)  than  did  their  OTS  counterparts.  In  addition,  the  ROTC 
subjects  were  less  accurate  on  this  test  (M  ROTC  =  63.7%  correct,  M  OTS  =  67.2%  correct; 
t  [1053]  =  -5.30,  £  s  .01).  One  explanaTTon  for  the  group  differences  in  performance  on  this 
test  may  reside  in  the  nature  of  the  test  Items.  The  Self-Crediting  Word  Knowledge  Test  is 
essentially  a  vocabulary  test  in  which  the  subject  makes  predictions  about  his/her  performance. 
In  that  the  ROTC  subjects  had  lower  scores  on  the  AFOOT  vocabulary  subtest  (Word  Knowledge), 
their  poorer  performance  and  lower  expectations  on  the  Self-Crediting  Word  Knowledge  Test  are 
not  surprising.  If  self-confidence  levels  had  been  assessed  using  another  ability  domain,  the 
results  might  have  been  different. 
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Table  4.  Descriptive  Statistics  for  ROTC  BAT  Battery  Scores 
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Table  6.  Summary  of  BAT  Compariaona  between  ROTC 
and  OTS  Samplea 


Teat  acore 

ROTC  (N 
Mean 

=  380) 
SO 

OTS  (N 

Mean 

=  705) 
SD 

Two-tailed 

t-teat 

iWo-Hand  Coordination; 
X1  Tracking  Error 

11,646.4 

4,526.4 

11,531.8 

5,293.7 

0.35 

Complex  Coordination; 

X2  Tracking  Error 

9,497.5 

7,599.6 

8,421.0 

6,967.4 

2.29* 

Y2  Tracking  Error 

8,781.3 

9,258.2 

7,776.7 

8,559.4 

1.75 

Z2  Tracking  Error 

7,129.1 

6,154.6 

7,303.8 

7,000.3 

-0.40 

Encoding  Speed; 

Avg  RT  (ms)  -  correct 

responses 

738.0 

149.4 

743.9 

137.0 

-0.63 

Percent  Correct  (%) 

90.9 

4.7 

91.0 

4.4 

-0.37 

Avg  RT  X  %  Correct 

3,733.3 

8,242.6 

3,104.1 

7,462.9 

1.24 

Mental  Rotation; 

Avg  RT  (ms)  -  correct 

responses 

407.0 

211.3 

449.0 

268.0 

-2.56* 

Standard  Deviation  RT 
correct  res^jonses 

(ms)  - 

988.5 

283.2 

924.5 

333.5 

3.08** 

Percent  Correct  (%) 

90.2 

8.4 

90.7 

8.4 

-1.00 

Item  Recognition; 

Avg  RT  (ms)  -  correct 

responses 

861.6 

201.2 

868.1 

216.2 

-0.42 

Slope  (RT) 

-18.5 

20.6 

-18.3 

23.6 

-0.11 

Intercept  (RT) 

927.6 

229.4 

934.2 

252.0 

-0.37 

Percent  Correct  (%) 

94.3 

4.4 

95  1 

4.0 

-2.81** 

Time-Sharing; 

Slope  (Tracking  Difficuity) 

6.4 

11.1 

6.5 

9.9 

-0.13 

Intercept  (Tracking  Difficulty) 

294.3 

101.3 

283.6 

94.9 

1.69 

Avg  Tracking  Difficulty 

263.7 

36.5 

256.2 

35.9 

3.19** 

Avg  RT  (ms) 

1,202.8 

188.2 

1,238.5 

213.3 

-2.66** 

Avg  RT  X  Tracking  Difficulty  300,878.5 

55,924.8 

300,482.4 

59,546.2 

0.10 

Self-Crediting  Word  Knowledge; 

Avg  RT  (ms)  -  correct  responses 

7,812.2 

1 ,596.6 

7,592.5 

1,386.0 

2.30* 

Percent  Correct  (%) 

63.7 

10.1 

67.2 

10.0 

-5.30** 

Avg  RT  X  %  Correct 

-2,129.6 

16,333.5 

-3,365.1 

14,231.9 

1.26 

Bet 

38.0 

8.1 

40.2 

7.9 

-4.26** 

Activities  Interest  Inventory; 

N  High-Risk  Choices 

50.0 

9.2 

51.7 

9.7 

-2.66** 

Avg  RT  (ms) 

4,275.5 

905.2 

4,566.4 

964.7 

-4.71** 

Nof .  Thd  number  of  subjects  for  the  Item  Recognition  test  is  261  for  the  ROTC  group  and  697 


for  the  OTS  group. 
*p  ^  .05. 

^  .01. 
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Results  from  the  test  concerned  with  assessing  attitudes  toward  risk-taking  (Activities  Interest 
Inventory)  suggest  that  the  ROTC  subjects  were  quicker  to  make  decisions  (M  ROTC  =■  '!,275.5 
ms,  M  OTS  -  4,566.4  ms;  t  (1053)  =  -4.71,  £  si  ,01),  but  leas  willing  to  take  risks  (number 
of  high-risk  choices;  M  ROTC  =  50.0,  M  OTS  »  51.7;  t  [1053]  -  -2,66,  £  s  .01). 

In  most  Instances  where  there  was  a  statistically  significant  difference  in  BAT  performance 
between  the  two  groups,  the  size  of  the  difference  was  relatively  small  (differences  of  a  few 
milliseconds  in  response  time  or  less  than  1  percent  >n  response  accuracy).  Many  of  the  BAT 
score  comparisons  would  have  been  judged  non-significant  had  the  experiment-wise  error  rate 
been  limited  to  the  .05  level. 

If  BAT  scores  are  Incorporated  into  the  pilot  candidate  selection  process,  differences  In  BAT 
performance  between  the  ROTC  and  OTS  groups  would  not  affect  the  likelihood  of  selection  for 
pilot  training  for  a  particular  individual,  because-as  mentioned  earller-ROTC  and  OTS  pilot 
training  candidates  are  evaluated  by  separate  selection  boards. 


Factor  Structure 


A  factor  analysis  using  the  25  scores  from  the  BAT  battery  was  performed  to  evaluate 
relationships  among  the  eight  tests.  As  not  all  subjects  had  scores  on  the  Item  Recognition 
Test,  only  958  subjects  were  included  in  the  factor  analysis. 

The  BAT  score  correlation  matrix  shown  in  Table  7  indicates,  for  the  most  part,  that  performance 
measures  from  the  eight  tests  were  not  related  strongly  to  each  other.  The  strongest  "between-test" 
relationships  occurred  between  measures  of  the  same  type  (tracking  error  and  tracking  difficulty, 
response  times  from  the  cognitive  tests,  and  response  accuracy  scores  from  the  cognitive  tests). 
WIthIn-test  correlations  suggested  that  In  some  instances  two  or  more  scores  from  the  same 
test  were  redundant  (e.g..  Item  Recognition,  average  response  time  and  Intercept,  r  =  .94; 
Time-Sharing,  slope  and  Intercept,  r  =  -.72). 

A  principal  factors  analysis  was  conducted  using  the  25  BAT  scores.  Estimated  communalities 
of  several  measures  approached  or  exceeded  1.0,  which  suggested  a  high  degree  of  linear 
dependence  between  some  of  the  measures.  In  the  instances  where  two  or  more  derived  scores 
were  based  on  the  same  elements  (e.g.,  Item  Recognition  -  average  response  time,  slope  and 
intercept),  all  but  one  of  the  interrelated  scores  were  eliminated.  Because  of  this  redundancy, 
eight  of  the  25  BAT  scores  were  excluded  from  the  factor  analysis.  The  eliminated  scores 
included:  Encoding  Speed  -  response  time  by  percent  correct  Interaction;  Mental  Rotation  - 
response  time  standard  deviation;  Item  Recognition  -  slope  and  intercept;  Time-Sharing  -  slope 
and  Intercept;  and  response  time  by  tracking  interaction;  and  Self-Crediting  Word  Knowledge  - 
response  time  by  percent  correct  interaction.  The  resulting  principal  factors  analysis  yielded  six 
factors  with  eigenvalues  greater  tfian  1.0.  These  six  principal  factors  accounted  for  61.8%  of  the 
total  score  variance  among  the  17  test  scores  included  in  the  analysis.  These  factors  were 
rotated  both  orthogonally  by  the  Varimax  method  and  obliquely  by  the  Kaiser-Harris  method. 
The  two  methods  produced  very  similar  results,  which  differed  only  in  the  order  of  the  factors. 
Correlations  between  factors  in  the  oblique  solution  ranged  from  r  =  -.36  to  r  =  .19,  suggesting 
that  an  orthogonal  solution  would  be  appropriate. 

Table  8  shows  the  communalities  and  factor  loadings  for  the  final  orthogonal  rotation.  Factor 
I  was  defined  clearly  by  the  Complex  Coordination  tracking  error  scores.  Factor  II  appeared 
to  represent  finger  dexterity,  as  both  tracking  performance  and  response  latencies  loaded  heavily 
on  It.  Measures  for  Factors  III  and  V  represent  two  different  components  of  information  processing 
speed.  Factor  III  can  be  interpreted  as  a  perceptual  speed  factor  in  that  it  Is  defined  by  the 
response  latencies  from  the  three  cognitive/perceptual  abilities  tests  (Encoding  Speed,  Mental 
Rotation,  and  Item  Recognition).  Factor  V  reflects  verbal  information  processing,  in  that  the 
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Table  7.  Basic  Attributes  Test:  Test  Score  Correlation  Matrix 
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The  saiM>le  size  for  the  test  score  correlations  was  only  958.  as  subjects  without  Ite«  Recognition  Test  scores  were  not  Included.  Test 
Scores  1  through  25  are  presented  In  the  saw  order  as  BAT  scores  on  Tables  4-6.  Appendix  D  Includes  the  test  score  acronyw  and  their  wanlngs. 


Table  8.  Basic  Attributes  Test;  Summary  of  Varlmax  Rotated  Factor  Solution 


Test  score 

Commun 

allty 

1 

II 

Factor  loadings 
III  IV 

V 

VI 

TWo-Hand  Coordination; 

X1  Tracking  Error 

.456 

.253 

.611 

-.109 

-.039 

-.067 

-.033 

Complex  Coordination; 

X2  Tracking  Error 

.731 

.839 

.107 

.122 

-.002 

-.012 

-.019 

Y2  Tracking  Error 

.476 

.638 

.212 

.027 

-.124 

-.085 

-.015 

Z2  Tracking  Error 

.644 

.771 

.208 

.047 

.018 

.026 

-.060 

Encoding  Speed; 

Avg  RT 

.725 

.060 

.425 

.673 

-.036 

.143 

.257 

Percent  Correct 

.363 

.050 

-.017 

.154 

.016 

-.045 

.578 

Mental  Rotation; 

Avg  RT 

.417 

.079 

-.075 

.634 

-.027 

-.048 

-.003 

Percent  Correct 

.196 

-.120 

-.030 

-.134 

.158 

.076 

.364 

Item  Recognition; 

Avg  RT 

.490 

.071 

.482 

.490 

.017 

.042 

.104 

Percent  Correct 

.255 

-.024 

.002 

.063 

.005 

.054 

.497 

Time-Sharing 

Avg  Tracking  Difficulty 

.291 

-.194 

-.483 

-.113 

-.033 

.043 

.063 

Avg  RT 

.266 

.039 

.436 

.053 

-.024 

.264 

.038 

Self-Crediting  Word  Knowledge; 

Avg  RT 

.373 

-.028 

.087 

-.004 

-.292 

.521 

.093 

Percent  Correct 

.589 

-.019 

-.061 

-.021 

.740 

-.119 

.151 

Bet 

.400 

-.032 

.036 

-.001 

.614 

-.141 

1 

b 

o 

Activities  Interest  Inventory; 

N  High-Risk  Choices 

.046 

-.056 

-.142 

-.031 

.120 

-.071 

.045 

Avg  RT 

.529 

-.051 

.021 

-.007 

-.101 

-.718 

.017 

%  Of 

Cumulative 

explained 

%  explained 

Factor  1 

Eigenvalue  variance 

variance 

1 

2.61 

35.9 

35.9 

II 

1.48 

20.4 

56.3 

III 

1 

1.32 

18.2 

74.5 

IV 

0.71 

9.8 

84.3 

V 

0.69 

9.5 

93.8 

VI 

0.45 

6.2 

100.0 

Not#.  Th«  sample  size  for  the  factor  analysis  was  only  958,  as  subjects  without  Item  Recognition  Test 
scores  were  not  included. 
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response  latencies  from  the  two  verbally  dependent  tests  (Self-Crediting  Word  Knowledge  and 
Activities  Interest  Inventory)  were  the  primary  variables  contributing  to  It.  Factor  IV  represents 
self-assessment  and  self-confidence,  and  Factor  VI  represents  response  accuracy.  It  should  be 
noted  that  the  Self-Crediting  Word  Knowledge  and  Activities  Interest  Inventory  response  latencies 
loaded  In  opposite  directions  on  Factor  V,  even  though  they  are  positively  correlated  with  each 
other  (see  Table  7).  One  possible  reason  for  the  opposite  factor  loadings  Is  that  the  Self-Crediting 
Word  Knowledge  Test  score  reflects  verbal  reasoning  whereas  the  Activities  Interest  Inventory 
score  reflects  decisiveness. 

The  number  and  composition  of  the  factors  were  not  surprising,  as  the  original  BAT  battery 
was  designed  to  measure  individual  differences  in  three  broad  domains:  psychomotor  coordination, 
cognitive/perceptual  speed  and  accuracy  and  personality/attitudinal  characteristics.  To  a  large 
extent  the  six  factors  identified  reflect  these  ability  domains. 

These  results  suggest  that  a  factor  analytic  approach  may  be  useful  for  developing  BAT 
composite  Indices  similar  to  the  AFOOT  composites.  If  the  BAT  battery  becomes  an  operational 
selection  Instrument,  a  composite  measure  would  be  easier  to  interpret  than  17  summaiV  scores. 


IV.  CONCLUSION 

Test  score  profiles  are  needed  in  order  to  understand  the  performance  characteristics  of  test 
Instruments  and  interpret  test  scores.  In  particular,  score  profiles  are  useful  for  understanding 
the  nature  of  the  tests  and  the  abilities  being  measured,  detecting  score  irregularities,  and  making 
comparisons  between  applicable  target  groups.  However,  score  profiles  provide  these  benefits 
only  for  those  groups  on  which  they  are  based. 

As  previously  noted,  the  ROTC  and  OTS  score  profiles  presented  in  this  paper  should  not 
be  interpreted  as  normative  data  for  USAF  pilot  training  applicants,  due  to  the  fact  that  the 
subjects  used  in  this  study  had  already  been  preselected  for  pilot  training  on  the  basis  of  their 
academic  performance,  AFOOT  scores  and  Flight  Screening  Program  (FSP)  performance.  Further, 
these  score  profiles  contain  representatives  from  only  two  of  the  three  major  USAF  commissioning 
sources  (scores  from  US  Air  Force  Academy  [AFA]  cadets  were  not  available). 

The  eight  BAT  tests  described  in  this  report  are  being  considered  as  adjuncts  to  the  current 
USAF  pilot  candidate  selection  and  classification  procedure  (Pilot  Selection  and  Classification 
System,  or  PSACS).  If  PSACS  is  to  be  implemented  operationally,  true  BAT  score  norms  need 
to  be  developed  for  individuals  representing  the  ROTC,  OTS  and  AFA  commissioning  sources. 
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Table  A-1 .  Composition  of  AFOOT  Form  O  Subtests 


Subtest 

ffo  of 

items 

Descriptive  measures 

Verbal  Analogies 

25 

Ability  to  reason  and  recognize  relationships 
between  words 

Arithmetic  Reasoning 

25 

Ability  to  understand  arithmetic  relationships 
expressed  as  word  problems 

Reading  Comprehension 

25 

Ability  to  read  and  comprehend  paragraphs 

Data  Interpretation 

25 

Ability  to  Interpret  data  from  graphs  and  charts 

Word  Knowledge 

25 

Ability  to  understand  written  language  through 
use  of  synonyms 

Math  Knowledge 

25 

Ability  to  use  mathematical  terms,  formulas,  and 
relationships 

Mechanical  Comprehension 

20 

Mechanical  knowledge  and  understanding  of 
mechanical  functions 

Electrical  Maze 

20 

Spatial  ability  based  on  choice  of  a  correct  path 
through  a  maze 

Scale  Reading 

40 

Ability  to  read  scales  and  dials 

instrument  Comprehension 

20 

Ability  to  determine  aircraft  attitude  from 
Illustrations  of  flight  instruments 

Block  Counting 

20 

Spatial  ability  through  analysis  of  a  three- 
dimensional  representation  of  a  set  of  blocks 

Table  Reading 

40 

Ability  to  read  tables  quickly  and  accurately 

Aviation  Information 

20 

Knowledge  of  general  aeronautical  concepts  and 
terminology 

Rotated  Blocks 

15 

Spatial  aptitude  by  visualizing  and  manipulating 
objects  in  space 

General  Science 

20 

Knowledge  and  understanding  of  scientific  terms, 
concepts,  principles,  and  instruments 

Hidden  Figures 

15 

Visual  imagery  and  perceptual  ability  using  a 
simple  figure  embedded  in  complex  drawings 

20 


Table  A-2.  Subtest  Components  of  AFOOT  Form  O  Composite  Scores 


AFOOT  Subtests _ 

Verbal  Analogies 
Arithmetic  Reasoning 
Reading  Comprehension 
Data  Interpretation 
Word  Knowledge 
Math  Knowledge 
Mechanical  Comprehension 
Electrical  Maze 
Scale  Reading 
Instrument  Comprehension 
Block  Counting 
Table  Reading 
Aviation  Information 
Rotated  Blocks 
General  Science 
Hidden  Figures 
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APPENDIX  B:  DESCRIPTIONS  OF  TESTS  RECOMMENDED  FOR  ALTERATION 
OR  REMOVAL  FROM  THE  BAT  BATTERY 
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Introduction 


The  following  tests  were  Included  in  the  original  BAT  battery,  but  have  been  removed  due 
to  their  poor  predictive  validity  for  USAF  pilot  training  performance.  TWo  of  these  tests, 
Decision-Making  Speed,  and  the  Automated  Aircrew  Personality  Profiler  have  been  modified  and 
currently  are  being  evaluated  for  inclusion  In  the  BAT  battery. 


Dot  Estimation 


Two  boxes  containing  an  arbitrary  number  of  dots  are  presented  simultaneously  on  the 
screen.  One  of  the  two  boxes  contains  one  more  dot  than  the  other.  The  subject’s  task  is 
to  determine,  as  quickly  as  possible,  which  box  has  the  greater  number  of  dots.  The  subject 
is  not  told  to  count  the  dots  In  each  box,  but  Is  told  only  to  decide  as  quickly  and  accurately 
as  possible  which  box  has  the  greater  number. 

Reaction  time  and  accuracy  of  response  are  recorded  on  each  trial.  This  Is  the  only  test 
in  the  battery  that  has  a  fixed  time  limit  (5  minutes,  for  a  maximum  of  55  trials).  The  psychological 
factor  assessed  by  this  test  Is  Impulsiveness, 'decisiveness. 


Digit  Memory 

A  string  of  four  digits  is  presented  simultaneously  and  in  random  order  The  subject  is 
instructed  to  respond  by  entering  the  digit  string  on  a  data  entry  keypad  In  the  same  order  as 
presented.  In  addition  to  the  recording  of  response  accuracy  and  overall  response  time,  a 
measure  of  perceptual  speed  is  taken  by  forcing  the  subject  to  press  a  special  ENABLE  key 
that  activates  the  data  entry  keypad  buttons  on  each  trial. 

The  most  important  attribute  measured  by  this  test  Is  perceptual  speed.  There  are  20  trials, 
which  require  about  5  minutes  to  complete. 


Immedlate/Delaved  Memory 

In  this  test,  a  sequence  of  digits  is  presented,  and  the  subject  Is  required  to  respond  by 
indicating  the  digit  that  occurred  either  one  or  two  digits  previously.  The  one-back  and  two-back 
subtests  have  two  parts.  In  the  first  part,  the  digits  are  presented  for  1/2  second,  followed  by 
a  2-second  interstimulus  Interval.  In  the  socond  part,  the  Interstimulus  interval  is  5  seconds. 
Thus,  for  both  subtests,  part  one  deals  with  "immediate"  memory  and  part  two  with  "delayed" 
memory. 

There  are  25  trials  in  each  subtest  (one-  versus  two-back)  for  each  length  of  latency  condition 
(2  versus  5  seconds)  resulting  In  100  trials.  As  with  the  other  tests,  response  time  and  accuracy 
are  recorded  on  each  trial. 

This  test  assesses  continuous  short-term  memory  storage  and  retrieval  operations. 


Decision-Making  Speed 

In  this  choice-reactlon-time  test,  one  of  several  alternative  signals  Is  presented  to  the  subject. 
The  subject  Is  required  to  respond  to  the  signal  as  quickly  as  possible.  The  critical  manipulation 
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In  this  test  Is  the  amount  of  uncertainty  that  must  be  resolved  In  order  to  make  the  response 
decision.  When  an  Increased  number  of  potential  alternatives  are  Introduced,  greater  uncertainty 
exists  and  the  desislon  Is  made  more  slowly.  This  test  consists  of  four  subtests. 

In  subtest  one,  the  subject  knows  both  where  and  when  a  signal  Is  to  occur;  In  subtest 
two,  the  subject  knows  where  but  not  when;  In  subtest  three,  when  but  not  where;  and  finally. 
In  subtest  four,  the  subject  knows  neither  where  nor  when.  Each  subtest  has  three  parts.  In 
part  one,  two  potential  signals  and  responses  are  defined.  There  are  four  potential  signals  and 
responses  In  part  two  and  eight  potential  signals  and  responses  In  part  three.  Therefore,  degree 
of  uncertainty  of  signal  Is  manipulated  in  three  ways:  location  of  occurrence,  time  of  occurrence, 
and  number  of  signals/responses.  There  are  12  trials  within  each  part  of  each  subtest,  resulting 
In  144  trials  (12x3x4).  Response  time  and  accuracy  of  response  (correct/incorrect)  are  recorded 
for  each  trial. 

The  Decision-Making  Speed  Test  assesses  a  variety  of  psychological  factors.  These  Include 
simple  choice  reaction  time  under  varying  degrees  of  information  load  and  spatial  and  temporal 
uncertainty,  as  well  as  low-level  cognitive  and  high-level  sensory-perceptual  motor  involvement. 


Risk-Taking 

In  this  test,  10  boxes  are  presented  in  two  rows  of  five  boxes  each.  The  subject  is  told 
that  9  of  the  10  boxes  contain  a  reward,  whereas  one  of  the  boxes  is  a  "disaster"  box.  If  the 
selected  box  contains  a  payoff,  the  subject  Is  allowed  to  keep  it;  but  if  the  subject  chooses 
the  disaster  box,  all  of  the  payoff  earned  on  that  trial  is  lost.  The  average  number  of  boxes 
selected  provides  an  index  of  the  subject’s  tendency  to  take  risks  when  making  decisions. 

Response  time  per  choice  and  number  of  boxes  chosen  are  recorded  on  each  of  30  trials. 
Unknown  to  the  subject,  there  Is  no  "disaster  box"  (i.e.,  no  risk)  for  12  of  the  30  trials.  This 
method  was  used  In  order  to  obtain  a  clean  measure  of  risk-taking  behavior,  as  performance 
on  the  "disaster  box"  trials  may  be  affected  by  chance. 


Embedded  Figures 

A  simple  geometric  figure  and  two  complex  geometric  figures  are  presented  to  the  subject. 
The  subject's  task  is  to  decide  which  one  of  the  two  complex  figures  has  the  simple  figure 
within  It  and  to  Indicate  a  choice  by  pressing  the  button  corresponding  to  the  figure.  Speed 
and  accuracy  of  response  are  recorded  on  each  of  30  trials.  This  test  is  designed  to  assess 
the  psychological  factor  of  field  dependence/independence. 


Automated  Aircrew  Personality  Profiler 

This  is  a  questionnaire  that  examines  the  subject’s  attitudes  and  Interests.  The  subject  is 
given  66  questions,  each  requiring  a  choice  between  two  alternatives.  The  subject  Is  Instructed 
not  to  spend  time  pondering  responses,  but  to  give  the  first  natural  answer  as  it  comes.  The 
questionnaire  is  a  traditionally  formulated  personality  Inventory  specially  compiled  In  cooperation 
with  the  USAF  School  of  Aerospace  Medicine  and  targeted  for  aircrew  selection  and  classification. 
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APPENDIX  C:  ILLUSTRATIONS  OF  SAMPLE  BAT  TEST  ITEMS  AND  APPARATUS 
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FIGURE  C-2.  COMPLEX  COORDINATION  TEST 
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FIGURE  C-3.  ENCODING  SPEED  TEST 
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FIGURE  C-4.  MENTAL  ROTATION  TEST 
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FIGURE  C-5.  ITEM  RECOGNITION  TEST 


FIGURE  C-6.  TIME- SHARING  TEST 
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FIGURE  C-7.  SELF-CREDITING  WORD  KNOWLEDGE  TEST 
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FIGURE  C-8.  ACTIVmES  MTEREST  MVENTORY 
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Figure  C-9.  Basic  Attributes  Test  Station. 


APPENDIX  D:  SCORING  PROCEDURES  FOR  THE  BAT  BATTERY 
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Introduction 


Scoring  procedures  for  the  Basic  Attributes  Test  (BAT)  battery  rely  on  a  combination  of 
tracking  error  and  difficulty,  response  speed,  response  accuracy  and  response  choice,  and  in 
some  instances,  specially  derived  scores  (e.g.,  regression  slope  and  intercept,  interaction  terms). 
Details  regarding  scoring  procedures  for  the  eight  validated  BAT  pilot  candidate  selection  and 
classification  tests  are  provided  below. 


TWo-Hand  Coordination 


Normally,  this  test  would  provide  two  tracking  error  scores  (XI  tracking  error  [horizontal] 
and  Y1  tracking  error  [vertical]).  However,  due  to  an  error  In  the  scoring  algorithm,  the  Y1 
tracking  error  score  was  accumulated  incorrectly  for  most  of  the  subjects  in  this  study.  As  a 
result,  Y1  tracking  error  is  not  reported  In  the  data  summaries  in  this  report,  but  Is  described 
here  for  completeness: 

XI  Tracking  Error  (PS2X1).  Cumulative  tracking  error  for  the  X1  axis  (horizontal  displacement 

of  the  cross  from  the  target)  for  the  final  2  minutes  of  the  test  period. 

Y1  Tracking  Error  (PS2Y1).  Cumulative  tracking  error  for  the  Y1  axis  (vertical  displacement 

of  the  cross  from  the  target)  for  the  final  2  minutes  of  the  test  period. 

Complex  Coordination 

X2  Tracking  Error  (PS2X2).  Cumulative  tracking  error  for  the  X2  axis  (horizontal  displacement 
of  the  cross  from  the  center  of  the  screen)  for  the  final  2  minutes  of  the  test  period. 

Y2  Tracking  Error  (PS2Y2).  Cumulative  tracking  error  for  the  Y2  axis  (vertical  displacement 
of  the  cross  from  the  center  of  the  screen)  for  the  final  2  minutes  of  the  test  period. 

Z2  Tracking  Error  (PS2Z2).  Cumulative  tracking  error  for  the  Z2  axis  (displacement  of  the 
rudder  bar  from  the  center  point  at  the  bottom  of  the  screen)  for  the  final  2  minutes  of  the 
test  period. 


Encoding  Speed 

Several  of  the  tests  In  the  BAT  battery  rely  on  response  latencies  (In  milliseconds)  as  an 
indicator  of  test  performane.  The  standard  scoring  technique  for  tests  of  this  type  uses  data 
only  from  trials  that  were  answered  correctly  when  computing  summary  scores.  For  the  BAT 
battery,  this  procedure  Includes  the  following  tests;  Encoding  Speed,  Mental  Rotation,  Item 
Recognition,  Time  f  hering  and  Self-Crediting  Word  Knowledge. 

The  following  scores  are  used  to  evaluate  performance  on  the  Encoding  Speed  Test; 

Average  Response  Time  (ENCRT).  Average  response  time  in  milliseconds  based  on  all  trials 
answered  correctly. 

Percent  Correct  (ENCPER).  Percent  correct. 

Average  Response  Time  x  %  Correct  (ENCINT).  This  is  a  response  time  by  percent  correct 
Interaction  term;  [Subject’s  average  response  time  -  grand  mean  average  response  time]  x 
[Subject's  percent  correct  -  grand  mean  percent  correct]). 
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Mental  Rotation 


Average  Response  Time  (MRTRT).  Average  response  time  in  milliseconds  based  on  all  trials 
answered  correctly. 

Standard  Deviation  (MRTSD).  Standard  deviation  of  response  time  in  milliseconds  based  on 
all  trials  answered  correctly. 

Pen \.ny  Correct  (MRTPER).  Percent  correct. 


Item  Recognition 

Average  Response  Time  (ITMRT).  Average  response  time  in  milliseconds  based  on  all  trials 
answered  correctly. 

Slope  (ITMSLP).  This  score  represents  a  regression  slope  for  the  best-fitting  line  for  average 
response  time  to  digit  strings  of  differing  lengths  (1,  2,  3,  4,  S  or  6  digits). 

Intercept  (ITMICP).  This  score  represents  a  regression  intercept  for  the  best-fitting  line  for 
average  response  time  to  digit  strings  of  different  lengths  (1,  2,  3,  4,  5  or  6  digits). 

Percent  Correct  (ITMPER).  Percent  correct. 


Time-Sharing 

Slope  (Tracking  Difficulty)  (TMSSLP).  Average  tracking  difficulty  was  computed  for  each 
subject  during  each  minute  of  the  practice  trials  (minutes  1-10).  This  score  represents  a  regression 
slope  based  on  the  best-fitting  line  describing  the  eight  average  tracking  difficulty  scores  for 
minutes  3-10  of  this  test  (learning  rate  on  the  tracking  task). 

Intercept  (Tracking  Difficulty)  (TMSICP).  This  score  represents  a  regression  Intercept  based 
on  the  best-fitting  line  describing  the  eight  i -minute  average  tracking  difficulty  scores  for  minutes 
3-10  of  the  test. 

Average  Tracking  Difficulty  (TMSDIF).  Average  tracking  difficulty  achieved  during  minutes 
11-19. 


Average  Response  Time  (TMSRT).  Average  response  time  in  milliseconds  to  cancel  the  digits 
that  appear  during  the  dual-task  trials  (minutes  11-16). 

Average  Response  Time  x  Tracking  Difficulty  (TMSRTD).  This  is  a  response  time  by  tracking 
difficulty  interaction  term  based  on  performance  during  the  dual-task  trials  (minutes  11-16).  it  is 
generated  in  a  manner  similar  to  that  for  ENCINT  (see  above). 


Self-Crediting  Word  Knowledge 

Average  Response  Time  (WKART).  Average  response  time  in  milliseconds  based  on  all  trials 
answered  correctly. 

Percent  Correct  (WKAPER).  Percent  correct. 
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Average  Response  Time  x  Percent  Correct  (WKAINT).  This  is  a  response  time  by  percent 
correct  Interaction  term.  It  is  generated  In  a  manner  similar  to  that  for  ENCINT  (see  above). 

Bet  (WKABET).  This  score  is  the  average  bet  the  subject  made  prior  to  each  of  the  three 
blocks  of  trials.  It  reflects  the  subject’s  self-confidence  regarding  his/her  expected  performance 
on  this  test.  Higher  scores  reflect  greater  self-confidence. 


Activities  Interest  Inventory 

Number  of  High-Risk  Choices  (AIAHIR).  This  score  Indicates  the  number  of  high-risk  choices 
made  by  the  subject. 

Average  Response  Time  (AIART).  Average  response  time  across  all  81  trials  (in  milliseconds). 
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Table  D-1.  Reliability  Estimatee  for  BAT  Performance  Seorea 


Teat  acore 

N 

No  Of 
acorea* 

Cronbach’a 

alpha 

Reference 

Two-Hand  Coordination: 

XI  Tracking  Error 

233 

10 

.94 

Mercatante,  1988 

Complex  Coordination: 

X2  Tracking  Error 

233 

10 

.95 

Mercatante,  1988 

Y2  Tracking  Error 

233 

10 

.99 

Z2  Tracking  Error 

233 

10 

.94 

Encoding  Speed: 

Response  Time 

2,219 

96 

.96 

Carretta,  1988a 

Response  Accuracy 

2,219 

96 

.71 

Mental  Rotation: 

Response  Time 

1,685 

72 

.97 

Not  previously 

Response  Accuracy 

1,685 

72 

.90 

reported 

Item  Recognition: 

Response  Time 

1,500 

48 

.95 

Not  previously 

Response  Accuracy 

1,500 

48 

.54 

reported 

Time-Sharing: 

Tracking  Difficuity 

1,130 

19 

.96 

Carretta,  1987b 

Self-Crediting  Word  Knowiedge: 

Response  Time 

1,992 

30 

.89 

Carretta  &  Siam, 

Response  Accuracy 

1,992 

30 

.65 

1988 

Activities  Interest  Inventory: 

Response  Time 

1,992 

81 

.95 

Carretta  &  Siam, 

Response  Choice 

1,992 

81 

.86 

1988 

*For  the  Two-Hand  Coordination  Ttst, 

Complax  Coordination 

Taat  and  Tima-Sharing  Taat,  "Scoraa" 

rofer  to  sunimad  tracking  parformanca  ovar  tima. 

For  tha  othai 

taata,  "Scoraa" 

rafar  to  taat  itamt. 

•  U.  S,  COVERNMEHT  PRINTING  OFPICE:  1990 — 761-051/20012 
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These  score  differences  do  nol  influence  the  comparative  likelihood  of  selection  for  pilot 
training  for  members  of  the  two  groups,  as  ROTC  and  OTS  pilot  training  candidates  are  evaluated 
by  separate  selection  boards. 


Basic  Attributes  Test 


Descriptive  Measures 

For  all  of  the  tests  in  the  BAT  battery,  tracking  error  scores,  response  latencies,  and  response 
choice/accuracy  are  used  to  assess  individual  differences  in  performance.  These  types  of  scores 
tend  to  exhibit  extremely  skewed,  non-normal  distributions.  It  is  difficult  to  interpret  summary 
statistics  from  such  distributions  as  both  means  and  standard  deviations  tend  to  be  distorted 
by  extreme  scores.  To  reduce  this  effect,  all  BAT  scores  that  were  more  than  three  standard 
deviations  from  the  mean  were  recoded  to  be  exactly  three  standard  deviations  from  the  mean. 
In  most  instances  this  affected  only  a  few  scores;  however,  up  to  8.85%  of  the  scores  for  a 
test  were  affected.  Even  after  recoding,  some  distributions  were  skewed.  See  Tables  4  and  5 
for  derailed  descriptions  of  the  ROTC  and  OTS  BAT  score  distributions.  Estimates  of  the  internal 
consistency  of  the  test  items  are  provided  in  Appendix  D  (Table  D-1). 

ROTC  Versus  OTS  Comparisons 


Although  the  ROTC  and  OTS  groups  exhibited  differences  in  performance  on  the  BAT  battery, 
the  direction  of  the  differences  did  not  clearly  favor  one  group  over  the  other.  Table  6  summarizes 
comparisons  between  the  ROTC  and  OTS  group  mean  scores. 

The  two  groups  did  not  differ  in  a  consistent  manner  in  tracking  performance.  The  OTS 
group  had  marginally  lower  X2  tracking  error  scores  on  the  Complex  Coordination  Test  (M  ROTC 
=  9,497.5,  M  OTS  =  8,421.0;  t  (1053]  =  2.29,  g  <  .05)  However,  the  ROTC  group  performed 
at  a  higher  average  tracking  difficulty  on  the  compensatory  tracking  task  used  in  the  Time-Sharing 
Test  (M  ROTC  =  263.7,  M  OTS  =  256.2;  t  (1053)  =  3.19,  g  <  01). 

Results  from  the  other  six  tests  also  were  mixed.  OTS  subjects  made  quicker  responses 
on  the  Mental  Rotation  average  response  time  (M  ROTC  =  988.5  milliseconds  (ms),  M  OTS  = 
924.5  ms;  t  (1053)  =  3.08,  g  <  .01)  whereas  ROTC  subjects  were  quicker  on  the  Time-Sharing 
average  response  time  (M  ROTC  =  1,202.8  ms,  M  OTS  =  1,238.5  ms;  t  [1053]  =  -2.66,  g  < 
.01).  OTS  subjects  achieved  a  higher  level  of  accuracy  than  did  the  ROTC  subjects  on  three 
of  the  cognitive  abilities  tests  (Encoding  Speed,  Mental  Rotation  and  Item  Recognition),  although 
this  difference  was  statistically  significant  for  only  the  Item  Recognition  Test  (M  ROTC  =  94.3% 
correct,  M  OTS  =  95.1%  correct;  t  [957]  -  -2.81,  g  <  .01). 

On  the  two  personality-type  BAT  tests,  ROTC  subjects  required  more  time  to  make  decisions 
and  were  less  willing  to  take  risks  than  were  the  OTS  subjects.  For  instance,  on  the  Self-Crediting 
Word  Knowledge  Test,  a  test  of  self-confidence,  ROTC  subjects  took  longer  to  make  decisions 
(average  response  time;  M  ROTC  =  7,812.2  ms,  M  OTS  =  7,592.5  ms;  t  [1053]  =  2.30,  g 
<  .05)  and  had  lower  expectations  about  their  performance  (bet  less;  M  ROTC  =  38.0,  M  OTS 
=  40.2;  t  [1053]  =  -4.26,  g  <  '.01)  than  did  their  OTS  counterparts.  In  addition,  the  ROTC 
subjects  were  less  accurate  on  this  test  (M  ROTC  =  63.7%  correct,  M  OTS  =  67.2%  correct; 
t  [1053]  ---  -5.30,  g  <  .01).  One  explanation  for  the  group  differences  in  performance  on  this 
test  may  reside  in  the  nature  of  the  test  items.  The  Self-Crediting  Word  Knowledge  Test  is 
essentially  a  vocabulary  test  in  which  the  subject  makes  predictions  about  his/her  performance. 
In  that  the  ROTC  subjects  had  lower  scores  on  the  AFOOT  vocabulary  subtest  (Word  Knowledge), 
their  poorer  performance  and  lower  expectations  on  the  Self-Crediting  Word  Knowledge  Test  are 
not  surprising.  If  self-confidence  levels  had  been  assessed  using  another  ability  domain,  the 
results  might  have  been  different. 
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Table  6.  Summary  of  BAT  Comparisons  between  ROTC 
and  OTS  Samples 


Test  score 

ROTC  (Kl 

Mean 

=  350) 
SD 

OTS  (N 
Mean 

=  7.05) 
SD 

Two-tailed 

t-test 

Two-Hand  Coordination: 

XI  Tracking  Error 

11,646.4 

4,526.4 

11,531.8 

5,293.7 

0.35 

Complex  Coordination: 

X2  Tracking  Error 

9,497.5 

7,599.6 

8,421.0 

6,967.4 

2.29* 

Y2  Tracking  Error 

8,781.3 

9,258.2 

7,776.7 

8,559.4 

1.75 

Z2  Tracking  Error 

7,129.1 

6,154.6 

7,303.8 

7,000.3 

-0.40 

Encoding  Speed: 

Avg  RT  (ms)  -  correct  responses 

738.0 

149.4 

743.9 

137.0 

-0.63 

Percent  Correct  (%) 

90.9 

4.7 

91.0 

4.4 

-0.37 

Avg  RT  X  %  Correct 

3,733.3 

8,242.6 

3,104.1 

7,462.9 

1.24 

Mental  Rotation: 

Standard  Deviation  RT  (ms)  - 

correct  responses 

407.0 

21 1.3 

449.0 

268.0 

-2.56* 

Avg  RT  (ms)  -  correct  responses 

988.5 

283.2 

924.5 

333.5 

3.08** 

Percent  Correct  (%) 

90.2 

8.4 

90.7 

8.4 

-1.00 

Item  Recognition: 

Avg  RT  (ms)  -  correct  responses 

861.6 

201.2 

868.1 

216.2 

-0.42 

Slope  (RT) 

-18.5 

20.6 

-18.3 

23.6 

-0.11 

Intercept  (RT) 

927.6 

229.4 

934.2 

252.0 

-0.37 

Percent  Correct  (%) 

94.3 

4,4 

95.1 

4.0 

-2.81** 

Time-Sharing: 

Slope  (Tracking  Difficulty) 

6  4 

11.1 

6.5 

9.9 

-0.13 

intercept  (Tracking  Difficulty) 

294.3 

101.3 

283.6 

94.9 

1.69 

Avg  Tracking  Difficulty 

263.7 

36.5 

256.2 

35.9 

3.19** 

Avg  RT  (ms) 

1,202.8 

188.2 

1,238.5 

213.3 

-2.66** 

Avg  RT  X  Tracking  Difficulty 

300,878.5 

55.924.8 

300,482.4 

59,546.2 

0.10 

Self-Crediting  Word  Knowledge: 

Avg  RT  (ms)  -  correct  responses 

7,812.2 

1.596.6 

7.592.5 

1,386.0 

2.30* 

Percent  Correct  (%) 

63.7 

10.1 

67.2 

10.0 

-5.30** 

Avg  RT  X  %  Correct 

-2,129.6 

16,333.5 

-3,365.1 

14,231.9 

1.26 

Bet 

38.0 

8.1 

40.2 

7.9 

-4.26** 

Activities  Interest  Inventory: 

N  High-Risk  Choices 

50.0 

9  2 

51.7 

9.7 

-2.66** 

Avg  RT  (ms) 

4,275.5 

905.2 

4,566.4 

964.7 

-4.71** 

Note.  The  number  of  subjects  for  the  Item  Recognition  test  is  261  for  the  ROTC  group  and  697 


for  the  OTS  group. 
*p  <  .05. 

••p  <  .01. 
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